1. Field of the Invention
The present invention relates to a technique for correcting chromatic aberrations of an optical system and, more particularly, to a technique for correcting chromatic aberrations generated in an image capturing system or display system of an HMD (Head Mounted Display).
2. Description of the Related Art
In recent years, as a technique for seamlessly blending physical and virtual worlds in real time, mixed reality, that is, a so-called MR (Mixed Reality) technique is known. As one MR technique, the following technique is known. That is, using a video see-through HMD (Head Mounted Display; to be referred to as “HMD” hereinafter), an object which nearly matches an object observed from the pupil position of the user who wears the HMD (to be referred to as an HMD user hereinafter) is captured by a video camera or the like. By displaying an MR image obtained by superimposing a CG (Computer Graphics) image on that captured image on a display unit of the HMD, that image is presented to the HMD user.
The video see-through HMD has a configuration in which a charge coupled element such as a CCD or the like captures an image of an object to acquire digital image data of this object, and an MR image (mixed reality image) obtained by superimposing a CG image on that digital image data is displayed on a display device such as a liquid crystal display or the like.
Size and weight reductions of the HMD mounted on the head are demanded. As for image capturing and display optical systems, in place of correction of various aberrations by optical approaches that lead to increases in size and weight, electronic correction by signal processing is generally applied, thus allowing to adopt low-cost lenses or to reduce the number of lenses.
When the optical systems are configured using low-cost lenses or the number of lenses which do not suffice to apply sufficient correction, the high image qualities of a captured image and that to be displayed cannot often be maintained due to aberrations of lenses. That is, barrel- or pin-cushion-shaped images are often obtained due to distortion aberrations of lenses. Also, red, blue, and green color bleedings appear at the boundaries of object images due to chromatic aberrations of magnification of lenses. For this reason, a technique that corrects an image quality drop of an object image due to such aberrations of lenses is demanded.
Techniques which correct distortion aberrations and chromatic aberrations of magnification of various aberrations of the optical systems by signal processing are disclosed. Such techniques are roughly classified into the following three techniques based on their principal methods, and their overviews will be explained.
The first technique is correction processing of distortion aberrations and chromatic aberrations of magnification by means of address conversion. Patent reference 1 discloses correction of distortion aberrations, and patent reference 2 discloses a technique associated with correction of chromatic aberrations in combination with distortion aberrations.
The address conversion is a method of moving a distorted image to an ideal image position based on the correspondence between an imaging position obtained by an ideal optical system and an actual imaging position which suffers the influence of aberrations in the image capturing system. Various techniques ranging from that which stores the correspondence associated with converted positions as a table, and simply converts the correspondence (addresses) between the read and write addresses of a memory up to that which holds high-precision coordinate data after conversion are available. In the display system as well, a display position is converted based on the correspondence between a pixel to be displayed and actual display position. When such pixel conversion is done, correction of distortion aberrations can be implemented. When conversion is done for respective colors which define each pixel, correction of chromatic aberrations of magnification can be implemented.
The second technique is correction processing of chromatic aberrations of magnification by means of resolution conversion. Using different variable magnifications depending on colors, enlargement or reduction processing is applied to a source color, thus obtaining an image which suffers less color bleedings.
The third technique is correction processing of distortion aberrations using an approximate polynomial and of chromatic aberrations of magnifications by means of distortion aberration correction of respective colors. Approximation is made using a polynomial of high order including correction parameters as coefficients so as to calculate coordinates after conversion.
However, the aforementioned conventional techniques suffer the following problems.
In the address conversion, since the correspondence needs to be generally held as a reference table, the data size of the table becomes huge with increasing resolution of the image capturing system and display system. In particular, upon execution of correction of chromatic aberrations of magnification, if each pixel is defined by three components R, G, and B, a table with a data size three times that of a mere distortion aberration is required. A reference table may be decimated and required coordinates may be calculated by interpolation calculations. However, in consideration of the calculation precision and the circuit scale of the interpolation calculations, a great reduction of the table size cannot be expected. An increase in data size raises the access frequency to a memory device that configures the table, and implementation of a faster, larger-capacity memory is indispensable. This also raises the threshold at the time of implementation in terms of both cost and implementation.
Correction of chromatic aberrations by means of the resolution conversion is effective for an ideal optical system of a rotation-symmetry system configured by a single lens. However, in an optical system configured by several lenses to correct various optical aberrations, high positional precision of coordinates after conversion cannot be obtained, and optical systems to which such correction is applied are limited very much, as described above. Upon adopting a prism having free-form surfaces, which realizes a compact, lightweight optical system, an optical origin is deviated from an image center, thus often generating asymmetric aberrations in the up and down or right and left directions, and simple enlargement or reduction can hardly cope with such aberrations.
The correction processing using an approximate polynomial has to raise the order so as to improve the precision, and complicated calculation processing including multiplications and divisions results in an increase in circuit scale. In an optical system which is not a rotation-symmetry system, a plurality of polynomials are required, and it is troublesome to apply such correction while maintaining high precision.
Also, patent references 7 and 8 disclose a configuration which reduces the number of memories for reference tables by executing conversion with high precision for one of three primary colors R, G, and B, and calculating differences from the reference color for the remaining colors upon address conversion.    [Patent Reference 1] Japanese Patent Laid-Open No. 5-207351    [Patent Reference 2] Japanese Patent Laid-Open No. 6-292207    [Patent Reference 3] Japanese Patent Laid-Open No. 6-205273    [Patent Reference 4] Japanese Patent Laid-Open No. 11-161773    [Patent Reference 5] Japanese Patent Laid-Open No. 2004-234379    [Patent Reference 6] Japanese Patent Laid-Open No. 2004-336106    [Patent Reference 7] Japanese Patent Laid-Open No. 8-205181    [Patent Reference 8] Japanese Patent Laid-Open No. 2000-153323
Even when the method using the difference values, as disclosed in patent references 7 and 8, is used, its effect is limited to a reduction of the number of significant bits of an integer part, and a further reduction of the memory size is demanded while maintaining high calculation precision.